Minimum Description Length and Cognitive Modeling

Geometric Complexity - 1 Geometric Complexity and Minimum Description Length Principle

The question of how one should decide among competing explanations of data is at the heart of the scientific enterprise. Quantitative methods of selecting among models have been advanced over the years without an underlying theoretical framework to guide the enterprise and evaluate new developments. In this paper, we show that differential geometry provides a unified understanding of the model ...

Towards unsupervised learning of constructions from text

Statistical learning methods offer a route for identifying linguistic constructions. Phrasal constructions are interesting both from the viewpoint of cognitive modeling and for improving NLP applications such as machine translation. In this article, an initial model structure and search algorithm for attempting to learn constructions from plain text is described. An information-theoretic optimi...

Minimum Description Length Modeling of Etymological Data

The Minimum Description Length Principle in Coding and Modeling

We review the principles of Minimum Description Length and Stochastic Complexity as used in data compression and statistical modeling. Stochastic complexity is formulated as the solution to optimum universal coding problems extending Shannon’s basic source coding theorem. The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to ...

Model selection by normalized maximum likelihood

The Minimum Description Length (MDL) principle is an information theoretic approach to inductive inference that originated in algorithmic coding theory. In this approach, data are viewed as codes to be compressed by the model. From this perspective, models are compared on their ability to compress a data set by extracting useful information in the data apart from random noise. The goal of model...